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LISTING OF CLAIMS: 

Claim 1 (Cmrently Amended): An information input processing conqsuter system for mapping 
gestures to keys of a virtual invisible keyboard, the system comprising one or several cameras, 
one or more memories with CPU connected to the cameras, and processes running in the CPU 
that associate gesture movements made in a touch aonaor froe environment without to uching any 
touch sensors with typing and produce gesture associated textual output, wherein said processes 
capture gesture images, classify each type of gesture image into a respective one of a plurality of 
classes depending on the type of gesture, and associate each of the classes with one or more 
possible keys of the invisible keyboard, assign a probability to each of said possible keys, and 
integrate the probabilities assigned to the possible keys to identify a word for a sequence of 
gestures. 

Claim 2 (Original): The gesture-key mapping system as in claim 1» where a feedback is 
provided to the user on what kind of keys are associated with the user's gestures. 

Claim 3 (Original): The gesture-key mapping system as in claim 2, where the feedback is 
provided using one or more of the following: displaying keys on a display, playing sotmds labels 
for keys, displaying image indicators on a display, playing special sound indicators, projecting 
the keyboard to any surfece, and displaying picture of the keyboard with user's hands. 



3 G:\Ibm\l05\l3317\anicncf\133l7.8m6.doc 

PAGE 8120 ' RCVD AT 11312005 4:01 :39 PM [Eastern Standard Time] * SVR:USPT0{FXRF-1I8 ' DNIS:8729306 ' CSID:5167424366 ' DURATION (min-ss):04-42 



JflN-03-20^ 16=18 SCULLY SCOTT 5167424366 P. 

Claim 4 (Currently Amended): An infonnation input processing, gesture-key mapping computer 
system for generating text from hand gestures of a user relative to an invisible keyboard, the 
system comprising one or several cameras, one or more memories with CPU connected to the 
cameras, and processes running in the CPU that associate gesture movements with typing and 
produce gesture associated textual output, where the gesnire-key processing is provided using the 
following modules: 

a) a gesture capturing module that captures gestures, relative to an invisible 
keyboard and made ia- a touch - Qonoor fr e e environm e nt without tou chinfr any 
touch sensors, through camera sensors; 

b) a gesture classificator module that classifies each type of gesture into a 
respective one of a plurality of classes depending on the type of gesture 
movements; 

c) an associator module for associating gesture classes or sequence of gesture 
classes with one or more possible keys of the invisible keyboard and assigns a 
probability to each of said possible keys; and 

d) an integrator module that integrates the probabilities assigned to the possible 
keys to identify a word for a sequence of gestures. 

Claim 5 (Original): The system as m claim 4, where the integrator module includes one or more 
of the following: 

a) language module component that estimate probabilities of word strings 

corresponding to key candidate sequences; 
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b) character fiequency module that estimate probabilities of character strings 
corresponding key candidate sequences; 

c) confusable matrix that estimate how often correct gesture classes are 
confusable with another gesture classes; 

d) gesture classes probability model that estimate probability of observing a string 
of gesture classes given a sequence of gesture frames; 

e) computation of a probability of production a sequence of keys given a string of 
gesture frames; 

£) generation of a lattice of sequences of keys given sequence of gesture frames; 

g) finding the most probable sequence of keys from the lattice of key candidate 

strings. 



Claim 6 (Original): A system according to Claim 5, wherein each sequence of keys receives a 
probability score, and the sequences of keys that receive low scores are removed and are not 
continuing when new candidates for keys arrive. 



Claim 7 (Previously Presented): A system according to Claun 4» further comprising a gesture 
correlator module to allow to adjust automatically the invisible keyboard to changes in the hand 
positions of the user, and wherein the gesture correlator module acts between the gesture 
classificator module and the associator module to maintain a consistent moping of gesture 
classes to keys of the invisible keyboard despite said changes in the hand positions of the user. 
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Claim 8 (Currently Amended): The method for producing a textual output in which a user makes 
typing like gesture relative to a virtual, invisible keyboard made in a touch s e nsor fre e 
onvironmBHt \yithout toucbinfi any touch sensors and without the presence of a real keyboard and 
the gestures are associated with the most probable keys that would be typed if a keyboard were 
presented, said method including the steps of using a computer system to map gestures made, m 
a touch sensor fr ee e nvironmcmt without touching any t ouch sensors, to keys of the virtual 
keyboard, including the steps of running processes on the computer to capture gesture images, to 
classify each type of gesture image into a respective one of a plurality of classes depending on 
the type of gesture, and to associate each of the classes with one or more possible keys of the 
invisible keyboard, assign a probability to each of said possible keys, and mtegrate the 
probabilities assigned to the possible keys to identify a word for a sequence of gesttn-es. 

Claim 9 (Currently Amended): The method for producing a textual output in which a user makes 
typing like gestures relative to a virtual, invisible keyboard and without the presence of a real 
keyboard and the gestures are associated with the most probable keys that would be typed if a 
keyboard were presented, said method including the step of using a computer system to map 
gestures made^ in a touch sensor fr ee environm e n t without touching any t ouch sensors, to keys of 
the virtual keyboard, including the step of running processes on the computer to capture gesture 
images, to classify each type of gesture image into a respective one of a plurality of classes 
depending on the type of gesture, and to associate each of the classes with one or more possible 
keys of the invisible keyboard, assign a probability to each of said possible keys, and integrate 
the probabilities assigned to the possible keys to identify a word for a sequence of gestures, and 
wherein the probability is computed using HMM- 
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Claim 10 (Currently Amended): A method of typing using a virtual keyboard having a multitude 
of virtual keys, comprising the steps; 

making typing gestures relative to a virtual, invisible keyboard made in a touob - 
s e n s or fr ee environment without touching anv touch sensors and without any real keyboard; 

sensing die typing gestures; and 

producing, from the sensed typing gestures, gesture associated textual output 
including the step of running processes on a computer to capture gesture images, classify each 
type of gesture image into a respective one of a plurality of classes depending on the type of 
gesture, and to associate each of the classes with one or more possible keys of the invisible 
keyboard, assign a probability to each of said possible keys, and integrate the probabilities 
assigned to the possible keys to identify a word for a sequence of gestures. 

Claim 1 1 (Original): A method according to Claim 10, wherein the typing gestures are made by 
a person, and further comprising the steps of providing feedback to the person on texture output 
associated with the gestures. 

Claim 12 (Original): A method accordmg to Claim 1 1, wherein the step of providing feedback 
includes the step of displaying an image of typing keys associated with the gestures. 

Claim 13 (Cancelled). 
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Claim 14 (Currently Amended): A method of typing using a virtual invisible keyboard, 
comprising the steps: 

making typing gestures relative to a virtual, invisible keyboard made in - a touch 
s ensor fr ee e moronmont without touching any touch sensors and without any real keyboard; 
sensing the typing gestures; and 

producing, from the sensed typing gestures, gesture associated textual output: and 
wherein the producing step includes the steps of classifying each type of gesture into a respective 
one of a plurality of classes depending on the type of gesture, associating each of said classes 
with one or more possible keys of the mvisible keyboard, assigning a probability to each of said 
possible keys, and integrating the probabilities assigned to the possible keys to identifying a 
word for a sequence of gestures. 

Claim 15 (Original): A method according to Claim 14, wherein the producing step further 
includes the step of associating gesture classes with individual typing keys. 

Claim 16 (Previously Presented): A method according to Claim 14, further comprising 
providing tmining data in words or sentences with certain timing data. 

Claim 17 (Currently Amended): A typing system using a virtual, invisible keyboard, comprising 
means for sensing typing gestures made relative to a virtual, invisible keyboard made in a touch 
oenaor froc ^ e nvironm e nt without touching any touch sensors and without any real keyboard; and 
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means for producing, from the sensed typing gestures, gesture associated textual 
output said producing means including a computer and processes running on the computer to 
capture gesture images, to classify each type of gesture image into a respective one of a plurality 
of classes depending on the type of gesture, and associate each of the classes with one or more 
possible keys of the invisible keyboard, assign a probability to each of said possible keys, and 
integrate the probabilities assigned to the possible keys to identify a word for a sequence of 
gestures. 

Claim 1 8 (Original): A system according to Claim 17, wherein the typing gestures are made by a 
person, and further comprising means for providing feedback to the person on texture output 
associated with the gestures. 

Claim 19 (Original): A system according to Claim 18, wherein the means for providing 
feedback includes means for displaying an image of typing keys associated with the gestures. 

Claim 20 (Cancelled). 

Claim 21 (Currently Amended); A typing system using a virtual invisible keyboard, comprising 

means for sensing typing gestures made relative to a virtual, invisible keyboard 
made in a toiich se n s or fr ee environm egt without touching any touch sensors and without any 
real keyboard; and 

means for producing, from the sensed typing gestures, gesture associated textual 

output; and 
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wherein the producing means includes means for classifying each type of gesture 
into a respective one of a plurality of classes depending on the type of gesture, associating each 
of said classes with one or more possible keys of the invisible keyboard, assigning a probability 
to each of said possible keys, and integrating the probabilities assigned to die possible keys to 
identifying a word for a response of gestures. 

Claim 22 (Original): A system according to Claim 21, wherein the producing means further 
includes means for associating gesture classes with individual typing keys. 

Claim 23 (Previously Presented): A method according to Claim 10, wherein the step of running 
processes on the computer includes the step of mapping the sensed typing gestures to keys of the 
keyboard based on a statistical machine that interprets sequences of typing like gesture classes as 
intended words based on user typing gesture models. 

Claim 24 (Previously Presented): A method according to Claim 10, wherein gesture input is 
represented as wave forms that are digitized and clustered in gesture frames, and then processed 
by HMM machines that represent diflFerence models for different typing pattems; and the step of 
ruiming processes includes the step of using two statistical components, a first statistical 
component to map gestures to keys, and a second^ language model component to map keys to 
letters and words. 
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